Two-Stream Action Recognition-Oriented Video Super-Resolution

We study the video super-resolution (SR) problem for facilitating video analytics tasks, e.g. action recognition, instead of for visual quality. The popular action recognition methods based on convolutional networks, exemplified by two-stream networks, are not directly applicable on video of low spatial resolution. This can be remedied by performing video SR prior to recognition, which motivates us to improve the SR procedure for recognition accuracy. Tailored for two-stream action recognition networks, we propose two video SR methods for the spatial and temporal streams respectively. On the one hand, we observe that regions with action are more important to recognition, and we propose an optical-flow guided weighted mean-squared-error loss for our spatial-oriented SR (SoSR) network to emphasize the reconstruction of moving objects. On the other hand, we observe that existing video SR methods incur temporal discontinuity between frames, which also worsens the recognition accuracy, and we propose a siamese network for our temporal-oriented SR (ToSR) training that emphasizes the temporal continuity between consecutive frames. We perform experiments using two state-of-the-art action recognition networks and two well-known datasets--UCF101 and HMDB51. Results demonstrate the effectiveness of our proposed SoSR and ToSR in improving recognition accuracy.Comment: Accepted to ICCV 2019. Code: https://github.com/AlanZhang1995/TwoStreamS

GameFormer: Game-theoretic Modeling and Learning of Transformer-based Interactive Prediction and Planning for Autonomous Driving

Autonomous vehicles operating in complex real-world environments require accurate predictions of interactive behaviors between traffic participants. While existing works focus on modeling agent interactions based on their past trajectories, their future interactions are often ignored. This paper addresses the interaction prediction problem by formulating it with hierarchical game theory and proposing the GameFormer framework to implement it. Specifically, we present a novel Transformer decoder structure that uses the prediction results from the previous level together with the common environment background to iteratively refine the interaction process. Moreover, we propose a learning process that regulates an agent's behavior at the current level to respond to other agents' behaviors from the last level. Through experiments on a large-scale real-world driving dataset, we demonstrate that our model can achieve state-of-the-art prediction accuracy on the interaction prediction task. We also validate the model's capability to jointly reason about the ego agent's motion plans and other agents' behaviors in both open-loop and closed-loop planning tests, outperforming a variety of baseline methods

Evaluating Alzheimer's Disease Progression by Modeling Crosstalk Network Disruption

Aβ, tau and P-tau have been widely accepted as reliable markers for Alzheimer’s disease (AD). The crosstalk between these markers forms a complex network. AD may induce the integral variation and disruption of the network. The aim of this study was to develop a novel mathematic model based on a simplified crosstalk network to evaluate the disease progression of AD. The integral variation of the network is measured by three integral disruption parameters. The robustness of network is evaluated by network disruption probability. Presented results show that network disruption probability has a good linear relationship with Mini Mental State Examination (MMSE). The proposed model combined with Support vector machine (SVM) achieves a relative high 10-fold cross-validated performance in classification of AD vs normal and mild cognitive impairment (MCI) vs normal (95% accuracy, 95% sensitivity, 95% specificity for AD vs normal; 90% accuracy, 94% sensitivity, 83% specificity for MCI vs normal). This research evaluates the progression of AD and facilitates AD early diagnosis

Inspection of delamination defect in first wall panel of Tokamak device by using laser infrared thermography technique

First wall panels (FWPs), which adjoin the inner wall of the blanket modules in the vacuum vessel (VV) of a Tokamak device, are in structures of multilayer bounded together with a solid welding technique in order to perform its heat exchange, VV protection, and neutron breeding functions. The quality of the welding joint between layers is the key factor for FWP integrity. In order to conduct online inspection of the delamination defect in the FWPs, a nondestructive testing (NDT) method capable to detect delamination defect without accessing into the VV is required. In this paper, the feasibility of the laser infrared thermography (LIRT) testing NDT method was investigated experimentally for this purpose. To clarify its detectability under practical VV environment, inspections of several inspection modes were conducted based on the practical structure of FWP and VV of the EAST Tokamak device, i.e., modes of different distances and angles of FWPs toward the LIRT transducers. In practice, an LIRT testing system was established and several double-layered plate specimens with different artificial delamination defects were inspected under the selected testing conditions. Through thermography signal reconstruction, an image processing algorithm was proposed and adopted to enhance the defect detectability. From the results of different inspection modes, it was found that the angle factor may worsen the inspection precision and reduce the detectability for delamination defects in case of big defect depth-to-width ratio, even though the LIRT method is still applicable for inspection of relative large defects in FWP. Finally, the detectability in different inspection modes was clarified, which proved the feasibility of LIRT for FWP online inspection

Progress on the ultrasonic testing and laser thermography techniques for NDT of tokamak plasma-facing components

During manufacturing and operation, different kinds of defects, e.g., delamination or surface cracks, may be generated in the plasma-facing components (PFCs) of a Tokamak device. To ensure the safety of the PFCs, various kinds of nondestructive testing (NDT) techniques are needed for different defect and failure mode. This paper gives a review of the recently developed ultrasonic testing (UT) and laser thermography methods for inspection of the delamination and surface cracks in PFCs. For monoblock W/Cu PFCs of divertor, the bonding quality at both W-Cu and Cu-CuCrZr interfaces was qualified by using UT with a focus probe during manufacturing. A noncontact, coupling-free and flexible ultrasonic scanning testing system with use of an electromagnetic acoustic transducer and a robotic inspection manipulator was introduced then for the in-vessel inspection of delamination defect in first wall (FW). A laser infrared thermography testing method is highlighted for the on-line inspection of delamination defect in FW through the vacuum vessel window of the Tokamak reactor. Finally, a new laser spot thermography method using laser spot array source was described for the online inspection of the surface cracks in FW